1. Field of the Invention
The present invention relates to a coding control method and a coding control apparatus for compression-coding a video signal or the like and transmitting the same in high efficiency.
2. Description of the Background Art
As the field of application of the technique of compression-coding a video signal in high efficiency and transmitting the same, a visual telephone or a video conference shown in FIG. 33A is general. Further, application to a system shown in FIG. 33B for transmitting a video signal through digital radio communication utilizing a transmission path of wireless LAN for monitoring a danger point or transmitting a picture between mobiles, and application to picture distribution utilizing Internet shown in FIG. 33C are expected.
With reference to video coding in a video conference or a visual telephone, the conventional video coding method is now described in detail.
Conventionally, in coding of a video conference or visual telephone signal, it is general to employ coding combining inter-picture coding utilizing inter-frame temporal correlation and intra-picture coding utilizing intra-frame spatial correlation with each other. A television image formed by 30 pictures (frames) per second has large correlation along the time axis direction, and if employing pixels on the same position of a screen precedent by one frame for prediction through inter-frame correlation, it follows that most ideal prediction can be performed when the screen is still. In INTER coding, however, inter-frame correlation contrarily lowers if there is motion in the screen, to be rather lower even as compared with correlation between adjacent pixels in a field. On the other hand, each pixel of a picture signal per frame has small level change with respect to an adjacent pixel and its correlation is strong. It is assumed that its self correlation function can be analogous to a negative exponential function. At this time, power spectral density which is Fourier transform of the self correlation function has a property of being maximized at a zero frequency component (i.e., dc component) and monotonously decreasing as the frequency component increases. While Fourier transform is best known as orthogonal transform to a frequency region, the Fourier transform includes complex number calculation and its structure is complicated, and hence it is general to employ two-dimensional DCT (Discrete Cosine Transform) in coding of pictures as substitute orthogonal transform. After a transform coefficient decomposed into frequency components by DCT is quantized to a level zero which is an uncoded transform coefficient (zero value of the coded coefficient) and a level xc2x11 to a level xc2x1K which are non-zero values of the coded coefficient (LEVEL) taking discrete quantization representative values, run-length coding for coding the number of successive zeros preceding the coded coefficient (RUN) and Huffman coding for allocating variable length codes in response to the originating rate of the level of the non-zero value of the coded coefficient (LEVEL) are performed, whereby video data are compressed.
For example, ITU-T recommendation H.261 applies motion compensation inter-picture coding to a picture having small motion while performing coding shown below on a prediction error between frames. Further, no inter-picture coding is applied to a picture having large motion but the following coding is directly performed on frame pixels. FIG. 31 shows an encoder and a decoder for video data according to H.261.
As shown in FIG. 31, an encoder 116 for video data according to H.261 comprises a subtraction part 107, a first orthogonal transform part 108 performing two-dimensional cosine transform, a first quantization part 109, a second inverse quantization part 110, a second inverse orthogonal transform part 111, an addition part 112, a second picture memory 113 for motion compensation, an in-loop filter 114, a coding control part 115 and selectors 123 and 124.
On the other hand, a decoder 122 comprises a first inverse quantization part 117, a first inverse orthogonal transform part 118, an addition part 119, a first picture memory 120 for motion compensation, an in-loop filter 121 and a selector 125.
The encoder 116 calculates by the subtraction part 107 a prediction error between frames by taking the difference between a video input signal previously transformed to CIF (Common Intermediate Format) of 352 by 288 dots and prediction data stored in the second picture memory 113 for motion compensation. At this time, motion in the range of 15 by 15 pixels is motion-compensated by specifying the prediction data as an arbitrary block of 16 by 16 pixels among 16 by 16 pixels around the block. The motion quantity is specified by a two-dimensional motion vector and transmitted to the decoder along with the video data. The decoding side decodes data of the picture memory for motion compensation in a region displaced from a decoding block by this motion vector as prediction data. For such large motion that no motion compensation is effective, INTRA coding with no prediction is selected by the selectors 123 and 124. The prediction error and the frame pixels are divided into blocks of 8 pixels by 8 lines, and two-dimensional cosine transform is performed on each block in the first orthogonal transform part 108. The pixels of each block are transformed to frequency components by the DCT. The obtained transform coefficients are quantized in the first quantization part 109. By the quantization, the respective transform coefficients are represented from the level 0 of the zero value of the coded coefficient to levels of non-zero values of the coded coefficient (LEVEL) which are integers up to a level xc2x1127. The quantized data, transmitted to the decoder through a communication part or the like, is inverse-transformed by the second inverse quantization part 110 and the second inverse orthogonal transform part 111 at the same time, thereafter added to the prediction data stored in the second picture memory 113 for motion compensation by the addition part 112, and stored in the second picture memory 113 for motion compensation to be next prediction data. The decoder 122 inverse-transforms the inputted video data through the first inverse quantization part 117 and the first inverse orthogonal transform part 118, thereafter adds the same to the prediction data stored in the first picture memory 120 for motion compensation through the adder 119, and obtains a video output while storing the same as next prediction data in the first picture memory 120 for motion compensation. When an input block is INTRA data, no prediction data is selected by the selector 125 but the input data is directly inverse-transformed, extracted as a video output, and stored in the picture memory for motion compensation.
The above is exemplary predictive coding of a video signal, particularly coding combining inter-picture coding and intra-picture coding. In INTER coding, mismatch is caused between the contents of frame memories of the coding side and the decoding side upon occurrence of a transmission error, and hence influence of the error propagates to all subsequent reproduced pictures. Therefore, it is necessary to transmit INTRA-coded video data for refreshing the reproduced pictures.
INTRA coding, which is coding utilizing no inter-frame correlation, has an enormous coding amount as compared with INTER coding. When transmitting a frame in which all blocks are INTRA-coded for refresh, therefore, it takes time for transmission and hence a delay time increases. In general, therefore, means of dividing one frame into a plurality of groups of blocks and refreshing a group of blocks every frame by INTRA coding thereby reducing increase of the coding amount per frame is considered.
The conventional coding control method in the coding control part 115 is now described. Spatial correlation or temporal correlation of a picture changes from moment to moment, and hence an information amount resulting from removing redundancy by DCT or inter-frame difference also changes from moment to moment. The information amount after coding can be controlled by changing the quantization characteristic in quantization of information resulting from performing DCT. However, compression of information by quantization is accompanied with distortion of the picture resulting from a quantization error. Coding control is achieved by selecting the quantization characteristic for approximating a generated information amount to a desired value. If the transmission rate is constant, redundancy fluctuating in response to the characteristic of an input picture is predicted to select the quantization characteristic so that the generated information amount is regularly uniform. For example, in a report xe2x80x9cA Study on Low Delay Interframe Prediction Codingxe2x80x9d announced in Proceedings of the 1992 Autumn Conference of the Institute of Electronics, Information and Communication Engineers, D-162 (1992) by Senda et al., there is disclosed a technique of coding amount control dividing a picture frame into blocks of 16 by 16 pixels called macroblocks and deciding, on the basis of the difference between an information amount generated before reaching an i-th macroblock and its predicted value, the quantization characteristic for a macroblock to be subsequently coded every time a macroblock is coded. In this report, the quantization characteristic (quantization parameter) for the next macroblock is decided by a target generated information amount in units of the macroblocks and an actually generated information amount thereby performing control of approximating a generated information amount of one picture frame to a target generated information amount of one picture frame. The processing is simple when uniformalizing the target generated information amount in units of the macroblocks in frames, while, when setting a macroblock having high inter-frame correlation and no such macroblock at the same target generated information amount, there arises such a problem that the picture quality of a block having large motion is deteriorated. In JP-A-5-344493, on the other hand, there is disclosed a motion picture coding apparatus which decides a coding parameter on the basis of an output from a predictive coding efficiency calculator calculating inter-picture coding efficiency and thereby performing optimum coding every part of a motion picture for solving this problem. However, it is not easy to decide a target generated information amount every macroblock responsive to inter-frame correlation while suppressing an information amount per frame within the target generated information amount but complicated processing follows.
In this regard, an operation of deciding the quantization characteristic with reference to a coding parameter control table previously decided on the basis of a generated information amount or an occupied amount of a buffer resulting from coding precedent by one frame is generally performed. However, the relation between the generated information amount and the quantization characteristic is not univocally decided and hence, as disclosed in JP-A-4-57489, having a coding parameter control table for each of previously assumed scene types is considered. Further, the applicant has proposed in JP-A-7-107482 a coding control method not univocally calculating the quantization characteristic on the basis of a generated information amount or an occupied amount of a buffer resulting from coding precedent by one frame but previously setting an upper bound and a lower bound of a target generated information amount for the quantization characteristic, also coding a next frame with the same quantization characteristic if a generated information amount resulting from coding one frame with a certain quantization characteristic is between the upper bound and the lower bound of the target generated information amount, and changing the quantization characteristic in a direction reducing quantization precision (i.e., in a direction deteriorating the picture quality) if the generated information amount resulting from coding one frame exceeds the upper bound of the target generated information amount, while changing the quantization characteristic in a direction raising the quantization precision (i.e., in a direction improving the picture quality) if the generated information amount resulting from coding one frame is below the lower bound of the target generated information amount. This gazette is incorporated herein by reference. According to this coding control method, it is possible to suit all scenes by simple control without defining an absolute level. With reference to FIG. 7A, the operation is now described.
It is assumed that the vertical axis in FIG. 7 shows a generated information amount, and the upper direction of this vertical axis is a direction increasing the generated information amount. It is also assumed that the horizontal axis in FIG. 7 shows quantization characteristics (QUANT), largeness/smallness of the quantization characteristics corresponds to largeness/smallness of quantization step sizes, and the right direction of this horizontal axis is a direction increasing the quantization step sizes (quantization errors greater). When regularly transmitting a picture of 30 frames per second under a constant transmission rate, it is ideal to render a generated information amount per frame transmission rate/30 frames. In practice, however, it is impossible to correctly control the generated information amount, and hence generated information amount taking a proper margin into consideration is set as a target generated information amount. It is assumed here that the quantization characteristics are expressed as q(s) for identifying each quantization characteristic by an index s which is a natural number, and the quantization step size of the quantization characteristic q(s) increases as the index s increases. At this time, the state of the quantization characteristic q(s) for a frame to be coded is specified by the value of the index s. It is hereinafter assumed that the state of the quantization characteristic for the frame to be coded is also expressed as q(s). In the coding control method disclosed in JP-A-7-107482, an upper bound threshold and a lower bound threshold of a generated information amount are decided every state of the quantization characteristic. These are decided to hold the target generated information amount therebetween. If a generated information amount resulting from coding a certain frame in a certain state q(s) is not in excess of the threshold (a), this state is left intact and coding of the next frame is also performed in the state q(s). If the generated information amount resulting from coding the certain frame in the certain state q(s) exceeds the upper bound threshold (b), transition is made to a state of a large quantization characteristic (a state of a large quantization step size), and as a result of coding the next frame with a quantization characteristic q(s+1), the next generated information amount is (c). If the generated information amount resulting from coding the certain frame in the certain state q(s) is below the lower bound threshold (d), transition is made to a state of a small quantization characteristic (a state of a small quantization step size), and as a result of coding the next frame with a quantization characteristic q(sxe2x88x921), the next generated information amount is (e).
The aforementioned coding control method controlling the quantization characteristic every frame has the following problem:
The quantization characteristic is a set of predetermined discrete quantization representative values. When quantizing one frame entirely with one quantization characteristic, errors are accumulated by the number of macroblocks forming one screen to result in difference in information amount, and hence the information amount largely fluctuates even if the quantization characteristic is most finely controlled. Thus, the control is accompanied with large errors. Particularly when using the coding control method proposed by the applicant, i.e., the coding control method not univocally calculating the quantization characteristic on the basis of a generated information amount or an occupied amount of a buffer resulting from coding precedent by one frame but deciding increase/decrease of the quantization characteristic on the basis of the generated information amount or the occupied amount of the buffer resulting from coding precedent by one frame, the generated information amount falls below the lower bound of the target generated information amount with the quantization characteristic q(s) while it exceeds the upper bound of the target generated information amount with the quantization characteristic q(sxe2x88x921) even if the correlation of the picture is constant, and hence such a situation may take place that the quantization characteristic fluctuates between q(s) and q(sxe2x88x921) every frame and the resulting generated information amount vibrates. In other words, such a situation may take place that the quantization characteristic fluctuates between two states corresponding to the respective ones of a point (exe2x80x2) and a point (d) shown in FIG. 7A and the generated information amount vibrates. FIG. 8A shows temporal transition of the generated information amount of the coding result at this time. In order not to exceed the target generated information amount even if the generated information amount is large, the upper bound of this vibration must be set smaller than the target generated information amount, and hence the generated information amount remarkably reduces on the lower bound of vibration and a transmission band cannot be efficiently used.
A generated information amount when coding a picture tends to increase as the motion of the object increases, and tends to increase if the area of the object is large even if the motion of the object is small. Further, when a background hidden behind the object appears due to motion of the object, the generated information amount increases/decreases depending on the fineness of the pattern of the background. In addition, in such a motion picture that motion of the object or a moving area gradually reduces and inter-frame correlation gradually rises, the generated information amount tends to reduce, while, in such a motion picture that motion of the object or a moving area gradually increases, the generated information amount tends to increase. Thus, mainly the generated information amount is influenced by the magnitude of the motion of the object, the magnitude of the moving area and the pattern of the background, and hence it is extremely difficult to correctly control a generated information amount of a picture frame to be subsequently coded, on the basis of a generated information amount of a previously coded picture frame. Therefore, in the aforementioned conventional coding control disclosed in JP-A-7-107482, the upper bound threshold and the lower bound threshold are set taking a proper margin into consideration for the target generated information amount. However, it is extremely difficult to set an upper bound threshold and a lower bound threshold taking into consideration the magnitude of the motion of the object, the magnitude of the moving area, the speed of the object and the pattern of the background, as well as a motion start and a motion end of the object. Therefore, for example, such a situation may take place that a generated information amount has fallen below the lower bound threshold and hence the quantization characteristic has been set in a direction improving the picture quality but the generated information amount exceeds the upper bound threshold if performing coding with the quantization characteristic set in the direction improving the picture quality, or such a situation may also take place that a generated information amount has exceeded the upper bound threshold on the contrary and hence the quantization characteristic has been set in a direction deteriorating the picture quality but the generated information amount falls below the lower bound threshold value if performing coding with the quantization characteristic set in the direction deteriorating the picture quality.
Problems in the case of applying the conventional coding control method disclosed in JP-A-7-107482 in such a situation that the magnitude of motion of the object or a moving area gradually reduces or increases are now described with reference to FIG. 32.
First, problems in the case of applying the conventional coding control method in such a situation that inter-frame correlation gradually rises.
Problem 1: In the case (b) where the generated information amount resulting from performing coding in the certain state q(s) exceeds the upper bound threshold value, such a situation is assumed that the motion of the object or the moving area gradually reduces and the inter-frame correlation gradually rises. In this case, the generated information amount has exceeded the upper bound threshold, and hence the quantization characteristic is set at q(s+1) to code the next frame. However, when setting the quantization characteristic at q(s+1) although the inter-frame correlation gradually rises and the generated information amount tends to gradually reduce, there arises such a problem that, due to the synergistic effect of the quantization characteristic and the inter-frame correlation, the generated information amount falls below the lower bound threshold value (cxe2x80x2) to deteriorate the picture quality beyond necessity. Further, inter-frame difference between the picture coded with the quantization characteristic q(s+1) and deteriorated in picture quality and a picture to be coded is taken in the next frame, and hence the difference increases and compressibility lowers by the deterioration of the picture quality.
Problem 2: In the case (d) where the generated information amount resulting from performing coding in the certain state q(s) is below the lower bound threshold, such a situation is assumed that the motion of the object or the moving area gradually reduces and the inter-frame correlation gradually rises. Since the generated information amount is below the lower bound threshold, the quantization characteristic is set at q(sxe2x88x921) to code the next frame. However, the inter-frame correlation gradually rises and the generated information amount tends to reduce, and hence, if performing coding with the quantization characteristic q(sxe2x88x921), the generated information amount may again fall below the lower bound threshold or reach a value around the lower bound threshold (exe2x80x3). Consequently, there arises such a problem that a transmission band cannot be efficiently used in transmission of coded data of a motion picture.
Problems in the case of applying the aforementioned conventional coding control in such a situation that inter-frame correlation gradually lowers are now described.
Problem 3: In the case (d) where the generated information amount resulting from performing coding in the certain state q(s) is below the lower bound threshold, such a situation is assumed that the motion of the object or the moving area gradually increases and the inter-frame correlation gradually lowers. In this case, the generated information amount is below the lower bound threshold and hence the quantization characteristic is set at q(sxe2x88x921) to code the next frame. However, when setting the quantization characteristic at q(sxe2x88x921), although the inter-frame correlation gradually lowers and the generated information amount tends to increase, the generated information amount abruptly increases (exe2x80x2) due to the synergistic effect of the quantization characteristic and the inter-frame correlation, to cause frame skip.
Problem 4: In the case (b) where the generated information amount resulting from performing coding in the certain state q(s) exceeds the upper bound threshold, such a situation is assumed that the motion of the object or the moving area gradually increases and the inter-frame correlation gradually lowers. In this case, the generated information amount exceeds the upper bound threshold, and hence the quantization characteristic is set at q(s+1) to code the next frame. However, the inter-frame correlation gradually lowers and the generated information amount tends to increase, and hence, when performing coding with the quantization characteristic q(s+1), the generated information amount may again exceed the upper bound threshold or reach a value around the upper bound threshold value (cxe2x80x3). Consequently, there arises such a problem that frame skip is caused in a motion picture transmitted in a coded state.
The present invention has been proposed in order to solve the aforementioned problems, and an object thereof is to provide a coding control method and a coding control apparatus improving precision of coding control and reducing a margin of a target generated information amount to the utmost and thereby enabling efficient transmission of a coded image to reduce loss. More concretely, the present invention aims at improving the precision of control of a quantization characteristic performed every frame with a simple structure. Further, the present invention aims at improving the precision and the speed of coding control by taking into consideration change of the magnitude of motion of an object or a moving area.
The present invention has the following features to solve the problems above.
A first aspect of the present invention is directed to a coding control apparatus for deciding, in coding data formed by a plurality of frames, on the basis of a generated information amount which is the amount of coded data generated by coding one frame or an occupied amount in a smoothing buffer used for transmission of data after coding, a quantization characteristic employed for coding a next frame to the one frame, comprising:
a comparison part for comparing the generated information amount or the occupied amount with a predetermined value, and
a quantization characteristic decision part dividing each frame of the plurality of frames previously into a plurality of areas while classifying the plurality of areas in each frame into a first group and a second group for setting in response to a comparison result by the comparison part the same quantization characteristic for the first and second groups or setting one of most approximate two different quantization characteristics for the first group and the other one for the second group respectively.
As described above, in the first aspect, the quantization characteristic decision part sets one of most approximate two quantization characteristics q(s) and q(s+1) for the first group while setting the other one for the second group, whereby a quantization state corresponding to an intermediate quantization characteristic between q(s) and q(s+1) is created, and hence with a simple algorithm more fine coding control is enabled.
According to a second aspect, in the first aspect,
the plurality of areas are so classified that the amount of coded data obtained by coding the first group and the amount of coded data obtained by coding the second group balance with each other previously into the first and second groups.
As described above, in the second aspect, the plurality of areas in each frame are previously classified into the first and second groups so that the respective coded data balance with each other, whereby precision of coding control can be improved without performing complicated analysis during the control.
According to a third aspect, in the second aspect,
the areas belonging to the first group and the areas belonging to the second group are spatially alternately arranged.
As described above, in the third aspect, the areas of the first group and the areas of the second group are spatially alternately arranged, whereby also in the case of coding data of a picture containing objects such as a background and a person having different properties a plurality of areas in its picture frame are classified into two groups uniformly.
According to a fourth aspect, in the second aspect,
the coding control apparatus further comprises:
a time axis placement control part for outputting a signal instructing an operation of exchanging a quantization characteristic to be set for the first group and a quantization characteristic to be set for the second group every prescribed time as a time axis placement control signal, and
the quantization characteristic decision part sets in response to the time axis placement control signal quantization characteristics for the first and second groups.
As described above, in the fourth aspect, the quantization characteristic employed for coding GOBs of the first group and the quantization characteristic employed for coding GOBs of the second group exchange every prescribed time, whereby distortion of a picture or non-uniformity of the picture quality resulting from spatial difference between the quantization characteristics can be rendered inconspicuous.
According to a fifth aspect, in the second aspect,
in the case that forced updating by INTRA coding is performed on a prescribed area among the plurality of areas when coding the data formed by the plurality of frames, the quantization characteristic decision part classifies, among the plurality of areas, an area to be subjected to forced updating to the first group while classifying the remaining areas other than the area to be subjected to forced updating to the second group.
As described above, in the fifth aspect, also when performing forced updating, a quantization state corresponding to an intermediate quantization characteristic between most approximate two quantization characteristics q(s) and q(s+1) can be created, whereby with a simple algorithm coding control of high precision is enabled.
According to a sixth aspect, in the fifth aspect, the quantization characteristic decision means sets quantization characteristics for the first and second groups so that quantization precision for coding areas to be subjected to forced updating among the plurality of areas is higher or equal to quantization precision for coding the remaining areas other than the areas to be subjected to forced updating.
As described above, in the sixth embodiment, when different quantization characteristics are set for the first and second groups, the quantization precision of an area to be INTRA-coded for forced updating is higher than the quantization precision of the other area to be INTER-coded, whereby block distortion due to INTRA coding is reduced.
According to a seventh aspect, in the first aspect,
the areas are groups of blocks.
As described above, in the seventh aspect, in transmission of coded data, single specification information may be transmitted every GOB as information specifying a quantization characteristic and no extra information may be transmitted, whereby compressibility improves.
According to a eighth aspect, in the first aspect,
the coding control apparatus further comprises:
a transition step width control part for comparing previously set upper bound and lower bound transition step width control thresholds with the generated information amount, wherein
a quantization characteristic set part sets, on the basis of a comparison result by the transition step width control part,
the quantization characteristic employed for coding the next frame so that, when the generated information amount is less than the upper bound transition step width control threshold and greater than the lower bound transition step width control threshold, only a quantization characteristic having been set for one of the first and second groups is changed or quantization characteristics for the first and second groups are maintained, and
sets the quantization characteristic employed for coding the next frame so that, when the generated information amount is greater than the upper bound transition step width control threshold or less than the lower bound transition step width control threshold, both the quantization characteristics having been set for the first and second groups are changed.
As described above, in the eighth aspect, in coding of picture data, when inter-frame correlation abruptly rises or lowers due to abrupt change of motion of an object or the like and hence abrupt increase/decrease of the generated information amount takes place, both the quantization characteristics for the first and second groups are so changed that the state (quantization state) of the quantization characteristic in the next frame is changed more greatly than general, whereby a delay of control resulting from performance of fine coding control can be prevented.
An ninth aspect of the present invention is directed to a coding control apparatus deciding in coding motion picture data formed by a plurality of frames on the basis of a generated information amount which is the amount of coded data generated by coding one frame or an occupied amount in a smoothing buffer used for transmission of data after coding a quantization characteristic employed for coding a next frame to the one frame, comprising:
a detection part for detecting, on the basis of a motion quantity of an object expressed by the motion picture data either one or both of a motion start state of the object and a motion end state of the object, and
a quantization control part for deciding, on the basis of the generated information amount or the occupied amount and a detection result by the detection part, the quantization characteristic employed for coding the next frame.
As described above, in the ninth aspect, on the basis of detection of the motion start state and/or the motion end state of the object, the quantization characteristic is controlled in response to the motion characteristic of the object, whereby the precision of coding control can be improved, and a time required until a proper quantization characteristic is set can be reduced (i.e., the speed of coding control can be improved).
According to a tenth aspect, in the ninth aspect,
the quantization control part suppresses, on the basis of the detection result by the detection part, change of a quantization characteristic employed for frame coding in the motion start state of the object in a direction improving the picture quality.
As described above, in the tenth aspect, when the object is in the motion start state and hence the generated information amount resulting from coding increases, the quantization characteristic is not changed in the direction improving the picture quality, whereby fluctuation of the generated information amount is suppressed and coding control of excellent precision can be performed.
According to an eleventh aspect, in the ninth aspect,
the quantization control part suppresses, on the basis of the detection result by the detection part, change of a quantization characteristic employed for frame coding in the motion end state of the object in a direction deteriorating the picture quality.
As described above, in the eleventh aspect, when the object is in the motion end state and hence the generated information amount resulting from coding reduces, the quantization characteristic is not changed in a direction deteriorating the picture quality, whereby fluctuation of the generated information amount is suppressed and coding control of excellent precision can be performed.
According to a twelfth aspect, in the ninth aspect,
the quantization control part changes, in the case of changing the quantization characteristic in a direction deteriorating the picture quality, when the motion start state of the object is detected by the detection part, the quantization characteristic more largely than that when the motion start state of the object is not detected.
As described above, in the twelfth aspect, in the motion start state of the object the quantization characteristic must be changed in the direction deteriorating the picture quality, when the quantization characteristic is largely changed, whereby the time required until a proper quantization characteristic is set can be reduced and coding control quickly following motion of the object can be performed.
According to a thirteenth aspect, in the ninth aspect,
the quantization control part changes, in the case of changing the quantization characteristic in a direction improving, when the picture quality the motion end state of the object is detected by the detection part, the quantization characteristic more largely than that when the motion end state of the object is not detected.
As described above, in the thirteenth aspect, in the motion end state of the object, when the quantization characteristic must be changed in the direction improving the picture quality, the quantization characteristic is largely changed, whereby the time required until a proper quantization characteristic is set can be reduced and the picture quality can be quickly improved.
According to a fourteenth aspect, in the ninth aspect,
the coding control apparatus further comprises
a storage part for storing an upper bound threshold and a lower bound threshold for the generated information amount or the occupied amount, and
a threshold set part for setting, on the basis of the detection result by the detection part, the upper bound threshold in a period of the motion start state of the object at a value lower than the upper bound threshold in a period other than the period of the motion start state of the object, and
the quantization control part changes the quantization characteristic in a direction deteriorating the picture quality when the generated information amount or the occupied amount exceeds the upper bound threshold and changes the quantization characteristic in a direction improving the picture quality when the generated information amount or the occupied amount falls below the lower bound threshold.
As described above, in the fourteenth aspect, in the period of the motion start state of the object the upper threshold is set low so that, when the object is in the motion start state and hence the generated information amount increases, the quantization characteristic is readily changed in the direction deteriorating the picture quality, whereby coding control quickly following motion of the object can be performed.
According to a fifteenth aspect, in the ninth aspect,
the coding control apparatus further comprises
a storage part for storing an upper bound threshold and a lower bound threshold for the generated information amount or the occupied amount, and
a threshold set part for setting, on the basis of the detection result by the detection part, the lower bound threshold in a period of the motion start state of the object at a value lower than the lower bound threshold in a period other than the period of the motion start state of the object, and
the quantization control part changes the quantization characteristic in a direction deteriorating the picture quality when the generated information amount or the occupied amount exceeds the upper bound threshold and changes the quantization characteristic in a direction improving the picture quality when the generated information amount or the occupied amount falls below the lower bound threshold.
As described above, in the fifteenth aspect, in the period of the motion start state the lower bound threshold is set low so that, when the object is in the motion start state and hence the generated information amount increases, the quantization characteristic is hardly changed in the direction improving the picture quality, whereby fluctuation of the generated information is suppressed and coding control of excellent precision can be performed.
According to a sixteenth aspect, in the ninth aspect,
the coding control apparatus further comprises
a storage part for storing an upper bound threshold and a lower bound threshold for the generated information amount or the occupied amount, and
a threshold set part for setting, on the basis of the detection result by the detection part, the upper bound threshold in a period of the motion end state of the object at a value higher than the upper bound threshold in a period other than the period of the motion end state of the object, and
the quantization control part changes the quantization characteristic in a direction deteriorating the picture quality when the generated information amount or the occupied amount exceeds the upper bound threshold and changes the quantization characteristic in a direction improving the picture quality when the generated information amount or the occupied amount falls below the lower bound threshold.
As described above, in the sixteenth aspect, in the period of the motion end state of the object the upper bound threshold is set high so that, when the object is in the motion end state and hence the generated information amount reduces, the quantization characteristic is hardly changed in the direction deteriorating the picture quality, whereby fluctuation of the generated information is suppressed and coding control of excellent precision can be performed.
According to a seventeenth aspect, in the ninth aspect,
the coding control apparatus further comprises
a storage part for storing an upper bound threshold and a lower bound threshold for the generated information amount or the occupied amount, and
a threshold set part f or setting, on the basis of the detection result by the detection part, the lower bound threshold in a period of the motion end state of the object at a value higher than the lower bound threshold in a period other than the period of the motion end state of the object, and
the quantization control part changes the quantization characteristic in a direction deteriorating the picture quality when the generated information amount or the occupied amount exceeds the upper bound threshold and changes the quantization characteristic in a direction improving the picture quality when the generated information amount or the occupied amount falls below the lower bound threshold.
As described above, in the seventeenth aspect, in the period of the motion end state of the object the lower bound threshold is set high so that, when the object is in the motion end state and hence the generated information amount reduces, the quantization characteristic is readily changed in the direction improving the picture quality, whereby the picture quality can be quickly improved.
According to an eighteenth aspect, in the ninth aspect,
the detection part detects such a state that a motion quantity of the object is greater than a previously set threshold and the motion quantity of the object continuously increases as the motion start state of the object.
As described above, in the eighteenth aspect, the motion start state of the object can be detected through simple processing.
According to a nineteenth aspect, in the ninth aspect,
the detection part detects such a state that a motion quantity of the object is greater than a previously set threshold and the motion quantity of the object continuously reduces as the motion end state of the object.
As described above, in the nineteenth aspect, the motion end state of the object can be detected through simple processing.
According to a twentieth aspect, in the ninth aspect,
the motion quantity is the sum of absolute values of motion vectors which are coded data when performing motion prediction inter-frame differential coding.
As described above, in the twentieth aspect, it is not necessary to provide new processing for detection of the motion start state of the object and the motion end state of the object, whereby reduction of coding processing ability can be prevented while low power consumption and cost reduction can be attained.
A twenty-first aspect of the present invention is directed to a coding control apparatus for deciding, in coding data formed by a plurality of frames, on the basis of a generated information amount which is the amount of coded data generated by coding one frame or an occupied amount in a smoothing buffer used for transmission of data after coding, a quantization characteristic employed for coding a next frame to the one frame, comprising:
a comparison part for comparing the generated information amount or the occupied amount with a predetermined value, and
a quantization characteristic decision part dividing each frame of the plurality of frames previously into a plurality of areas while classifying the plurality of areas in each frame into a first group and a second group for setting in response to a comparison result by the comparison part the same quantization characteristic for the first and second groups or setting one of most approximate two different quantization characteristics for the first group and the other one for the second group respectively.
According to a twenty-second aspect, in the twenty-first aspect, the plurality of areas are classified previously into the first and second groups so that the amount of coded data obtained by coding the first group and the amount of coded data obtained by coding the second group can balance with each other.
According to a twenty-third aspect, in the twenty-second aspect, the areas belonging to the first group and areas belonging to the second group are spatially alternately arranged.
According to a twenty-forth aspect, in the twenty-second aspect, a quantization characteristic to be set for the first group and a quantization characteristic to be set for the second group are exchanged every prescribed time in the quantization characteristic decision step.
According to a twenty-fifth aspect, in the twenty-second aspect, in case that forced updating by INTRA coding is performed on any one area among the plurality of areas when coding the data formed by the plurality of frames, an area to be subjected to forced updating among the plurality of areas is classified to the first group, while the remaining areas other than the area to be subjected to forced updating is classified to the second group in the quantization characteristic decision.
According to a twenty-sixth aspect, in the twenty-first aspect, the areas are groups of blocks.
According to a twenty-seventh aspect, in the twenty-first aspect, the coding control method comprising:
a second comparison step of comparing previously set upper bound and lower bound transition step width control thresholds with the generated information amount; wherein
in quantization characteristic decision steps, on the basis of a comparison result by the second comparison step,
the quantization characteristic employed for coding the next frame is set so that, when the generated information amount is less than the upper bound transition step width control threshold and greater than the lower bound transition step width control threshold, only a quantization characteristic having been set for one of the first and second groups is changed or quantization characteristics for the first and second groups are maintained, and
the quantization characteristic employed for coding the next frame is set so that, when the generated information amount is greater than the upper bound transition step width control threshold or less than the lower bound transition step width control threshold, both the quantization characteristics having been set for the first and second groups are changed.
According to a twenty-eighth aspect, a coding control method for deciding, in coding motion picture data formed by a plurality of frames, on the basis of a generated information amount being the amount of coded data generated by coding one frame or an occupied amount in a smoothing buffer used for transmission of data after coding, a quantization characteristic employed for coding a next frame to the one frame, comprising:
a detection step of detecting, on the basis of a motion quantity of an object expressed by the motion picture data, either one or both of a motion start state of the object and a motion end state of the object; and
a quantization control step of deciding, on the basis of the generated information amount or the occupied amount and a detection result by the detection step, the quantization characteristic employed for coding the next frame.
According to a twenty-ninth aspect, in the twenty-first aspect, on the basis of the detection result by the detection step, change of a quantization characteristic employed for frame coding in the motion start state of the object in a direction improving the picture quality is suppressed in the quantization control step.
According to a thirtieth aspect, in the twenty-eighth aspect, on the basis of the detection result by the detection step, change of a quantization characteristic employed for frame coding in the motion end state of the object in a direction deteriorating the picture quality is suppressed in the quantization control step.
According to a thirty-first aspect, in the twenty-eighth aspect, in the case of changing the quantization characteristic in a direction deteriorating the picture quality, when the motion start state of the object is detected by the detection step, the quantization characteristic is changed more largely than that when the motion start state of the object is not detected in the quantization control step.
According to a thirty-second aspect, in the twenty-eighth aspect, in the case of changing the quantization characteristic in a direction improving the picture quality, when the motion end state of the object is detected by the detection step, the quantization characteristic is changed more largely than that when the motion end state of the object is not detected in the quantization control step.
According to a thirty-third aspect, in the twenty-eighth aspect, a coding control further comprising:
a first set step of setting an upper bound threshold and a lower bound threshold for the generated information amount or the occupied amount; and
a second set step of setting, on the basis of the detection result by the detection steps, the upper bound threshold in a period of the motion start state of the object at a value lower than the upper bound threshold in a period other than the period of the motion start state of the object, wherein
the quantization characteristic is changed in a direction deteriorating the picture quality when the generated information amount or the occupied amount exceeds the upper bound threshold and the quantization characteristic is changed in a direction improving the picture quality when the generated information amount or the occupied amount falls below the lower bound threshold in the quantization control step.
According to a thirty-fourth aspect, in the twenty-eighth aspect, a coding control method further comprising:
a first set step of setting an upper bound threshold and a lower bound threshold for the generated information amount or the occupied amount; and
a second set step of setting again, on the basis of the detection result by the detection step, the lower bound threshold in a period of the motion start state of the object at a value lower than the lower bound threshold in a period other than the period of the motion start state of the object, wherein
the quantization characteristic is changed in a direction deteriorating the picture quality when the generated information amount or the occupied amount exceeds the upper bound threshold and the quantization characteristic is changed in a direction improving the picture quality when the generated information amount or the occupied amount falls below the lower bound threshold in the quantization control step.
According to a thirty-fifth aspect, in the twenty-eighth aspect,a coding control further comprising:
a first set step of setting an upper bound threshold and a lower bound threshold for the generated information amount or the occupied amount; and
a third set step of setting again, on the basis of the detection result by the detection step, the upper bound threshold in a period of the motion end state of the object at a value higher than the upper bound threshold in a period other than the period of the motion end state of the object, wherein
the quantization characteristic is changed in a direction deteriorating the picture quality when the generated information amount or the occupied amount exceeds the upper bound threshold and the quantization characteristic is changed in a direction improving the picture quality when the generated information amount or the occupied amount falls below the lower bound threshold in the quantization control step.
According to a thirty-sixth aspect, in the twenty-eighth aspect, a coding control method further comprising:
a first set step of setting an upper bound threshold and a lower bound threshold for the generated information amount or the occupied amount; and
a third set step of setting, on the basis of the detection result by the detection step, the lower bound threshold in a period of the motion end state of the object at a value higher than the lower bound threshold in a period other than the period of the motion end state of the object, wherein
the quantization characteristic is changed in a direction deteriorating the picture quality when the generated information amount or the occupied amount exceeds the upper bound threshold and the quantization characteristic is changed in a direction improving the picture quality when the generated information amount or the occupied amount falls below the lower bound threshold in the quantization control step.
According to a thirty-seventh aspect, in the twenty-eighth aspect, such a state that a motion quantity of the object is greater than a previously set threshold and the motion quantity of the object continuously increases is detected as the motion start state of the object in the detection step.
According to a thirty-eighth aspect, in the twenty-eighth aspect, such a state that a motion quantity of the object is greater than a previously set threshold and the motion quantity of the object continuously reduces is detected as the motion end state of the object in the detection steps.
According to a thirty-ninth aspect, in the twenty-eighth aspect, the motion quantity is the sum of absolute values of motion vectors being coded data when performing motion prediction inter-frame differential coding.
According to a fortieth aspect, a storage medium containing a coding control program deciding, in coding data formed by a plurality of frames on the basis of a generated information amount which is the amount of coded data generated by coding one frame or an occupied amount in a smoothing buffer used for transmission of data after coding, a quantization characteristic employed in coding of a next frame to the one frame as a program executed in a computer unit, wherein the coding control program implementing on the computer unit an operating environment which includes:
a comparison step of comparing the generated information amount or the occupied amount with a predetermined value, and
a quantization characteristic decision step of dividing each frame of the plurality of frames previously into a plurality of areas, classifying the plurality of areas in each frame into a first group and a second group, and setting the same quantization characteristic for the first and second groups or setting one of most approximate two different quantization characteristics for the first group and the other one for the second group respectively, in response to a comparison result by the comparison step.
According to a forty-first aspect, a coding control program deciding, in coding motion picture data formed by a plurality of frames on the basis of a generated information amount which is the amount of coded data generated by coding one frame or an occupied amount in a smoothing buffer used for transmission of data after coding, a quantization characteristic employed in coding of a next frame to the one frame as a program executed in a computer unit, wherein the coding control program implementing on the computer unit an operating environment which includes:
a detection step of detecting, on the basis of a motion quantity of an object expressed by the motion picture data, either one or both of a motion start state of the object and a motion end state of the object, and
a quantization control step of deciding, on the basis of the generated information amount or the occupied amount and a detection result by the detection step, the quantization characteristic employed in coding of the next frame.
According to a forty-second aspect, in the ninth aspect, the motion quantity is the sum of macroblocks whose absolute values of motion vectors being coded data at the time of motion prediction inter-frame differential coding performed to the respective macroblocks exceed a previously set threshold.
As described above, in the forty-second aspect, it is not necessary to provide new processing for detection of the motion start state of the object and the motion end state of the object, whereby reduction of coding processing ability can be prevented and low power consumption and cost reduction can be realized.
According to a forty-third aspect, in the twenty-eighth aspect, the motion quantity is the sum of macroblocks whose absolute values of motion vectors being coded data at the time of motion prediction inter-frame differential coding performed to the respective macroblocks exceed a previously set threshold.
These and other objects, features, aspects and advantages of the present invention will become more apparent from the following detailed description of the present invention when taken in conjunction with the accompanying drawings.